Web-based Question Answering: Revisiting AskMSR

Web-based QA, pioneered by Kwok et al. (2001), successfully demonstrated the power of Web redundancy. Early WebQA systems, such as AskMSR (Brill et al., 2001), rely on various kinds of rewriting and pattern-generation methods for identifying answer paragraphs and for extracting answers. In this paper, we conducted an experimental study to examine the impact of the advance of search engine technologies and the growth of the Web, to such Web-QA approaches. When applying AskMSR to a new question answering dataset created using historical TREC-QA questions, we find that the key step, query pattern generation is no longer required but instead, deeper NLP analysis on questions and snippets remains critical. Based on this observation, we propose a Web-QA system that removes the query pattern generation step and improves answer candidate generation and ranking steps, and outperforms AskMSR by 34 points of MRR.